Skip to content

fix(otel): configurable baggage, pass-through + service spans, typed guardrails - #29263

Merged
yassin-berriai merged 1 commit into
litellm_fix/kind-ptolemy-CNozrfrom
claude/litellm-tracing-config-Sh4Lz
May 29, 2026
Merged

fix(otel): configurable baggage, pass-through + service spans, typed guardrails#29263
yassin-berriai merged 1 commit into
litellm_fix/kind-ptolemy-CNozrfrom
claude/litellm-tracing-config-Sh4Lz

Conversation

@yassin-berriai

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #28909 (typed semconv-aligned OpenTelemetry V2 instrumentation). This branch carries that work plus fixes for four problems found in it. All changes stay gated behind LITELLM_OTEL_V2.

1. Baggage allowlists are configurable (not hard-coded)

baggage_promoted_keys / baggage_metadata_keys previously had no env or config wiring. They are now configurable two ways:

  • Env vars (comma-separated): LITELLM_OTEL_BAGGAGE_PROMOTED_KEYS, LITELLM_OTEL_BAGGAGE_METADATA_KEYS. A NoDecode + field_validator(mode="before") lets a plain a,b,c string parse into a list (pydantic-settings would otherwise demand JSON).
  • config.yaml: callback_settings.otel.baggage_promoted_keys (a YAML list). These reach the config because the OpenTelemetryV2 constructor now builds OpenTelemetryV2Config(**kwargs) from the settings passed through.

2. Pass-through LLM-call span no longer missing

Pass-through endpoints dispatch their logging from a detached asyncio.create_task whose copied OTel context may no longer carry the server span, so the LLM-call span was dropped / orphaned. Fix:

  • The pass-through endpoint now threads the proxy server span as litellm_parent_otel_span in request metadata (matching what normal proxy routes already do).
  • _emit_llm_call falls back to that explicitly-threaded parent span when the ambient context has no recordable span. The normal path (ambient server span present) is unchanged.

3. Service spans no longer always missing

litellm/_service_logger.py only dispatched to loggers that were instances of the legacy OpenTelemetry. The V2 OpenTelemetryV2 is a plain CustomLogger, so it never matched and Redis/Postgres service spans were always dropped. The dispatch now resolves both the legacy and V2 OTel loggers (via a lazily-imported, SDK-optional helper), whether registered as an instance or as the "otel" string.

4. Guardrail span data is typed and provider-agnostic

GuardrailSpanData.from_logging_entry was untyped (Mapping[str, object]) and read ad-hoc bare keys (name/status/mode) that match no single schema. It now:

  • Takes the typed StandardLoggingGuardrailInformation and reads only its canonical, provider-agnostic keys.
  • Normalizes guardrail_mode (a GuardrailEventHooks enum, a list of them, or a GuardrailMode) to a stable string — e.g. "pre_call", not "GuardrailEventHooks.pre_call", and joins multiple modes.
  • Surfaces the additional provider-agnostic fields guardrail_id, policy_template, detection_method as span attributes.

Tests

  • New tests/test_litellm/integrations/otel/test_otel_v2_config_fixes.py — baggage env/config wiring, pass-through threaded-parent fallback (and ambient-wins precedence), guardrail mode normalization + typed metadata mapping.
  • Extended tests/test_litellm/test_service_logger.py — V2 logger is recognized by the service-span dispatch (instance, "otel" string, unrelated-callback no-op) and an end-to-end service span is emitted.
  • Full tests/test_litellm/integrations/otel/ suite passes; ruff / mypy / black clean on the changed files.

Generated by Claude Code

@CLAassistant

CLAassistant commented May 29, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@codecov

codecov Bot commented May 29, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.94595% with 3 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
litellm/_service_logger.py 85.71% 3 Missing ⚠️

📢 Thoughts on this report? Let us know!

@yassin-berriai
yassin-berriai changed the base branch from litellm_internal_staging to litellm_fix/kind-ptolemy-CNozr May 29, 2026 06:03
@yassin-berriai
yassin-berriai changed the base branch from litellm_fix/kind-ptolemy-CNozr to litellm_fix/export_server_otel_span May 29, 2026 06:03
@yassin-berriai
yassin-berriai changed the base branch from litellm_fix/export_server_otel_span to litellm_fix/kind-ptolemy-CNozr May 29, 2026 06:04
…guardrails

- Baggage allowlists are configurable via LITELLM_OTEL_BAGGAGE_PROMOTED_KEYS /
  LITELLM_OTEL_BAGGAGE_METADATA_KEYS env vars (comma-separated) and via
  callback_settings.otel.* in config.yaml, instead of being hard-coded.
- Pass-through LLM-call spans now nest under the proxy server span: the proxy
  threads the server span as litellm_parent_otel_span and the adapter falls
  back to it when the ambient context (a detached logging task) has lost it.
- Service spans (Redis/Postgres) now emit under LITELLM_OTEL_V2: the service
  logger dispatch recognizes the V2 OpenTelemetryV2 logger, which is a plain
  CustomLogger and not a subclass of the legacy OpenTelemetry.
- Guardrail span data is built from the typed, provider-agnostic
  StandardLoggingGuardrailInformation (normalizing enum/list guardrail_mode and
  adding guardrail_id / policy_template / detection_method) instead of assuming
  one provider's field shape.
@yassin-berriai
yassin-berriai force-pushed the claude/litellm-tracing-config-Sh4Lz branch from d2986fd to f7bcd08 Compare May 29, 2026 15:35
@yassin-berriai
yassin-berriai merged commit f7bcd08 into litellm_fix/kind-ptolemy-CNozr May 29, 2026
43 of 45 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants